Deep Learning-Based, Passive Fault Tolerant Control Facilitated by a Taxonomy of Cyber-Attack Effects

In the interest of improving the resilience of cyber-physical control systems to better operate in the presence of various cyber-attacks and/or faults, this dissertation presents a novel controller design based on deep-learning networks. This research lays out a controller design that does not rely on fault or cyber-attack detection. Being passive, the controller’s routine operating process is to take in data from the various components of the physical system, holistically assess the state of the physical system using deep-learning networks and decide the subsequent round of commands from the controller. This use of deep-learning methods in passive fault tolerant control (FTC) is unique in the research literature. The proposed controller is applied to both linear and nonlinear systems. Additionally, the application and testing are accomplished with both actuators and sensors being affected by attacks and /or faults

Application of Fuzzy State Aggregation and Policy Hill Climbing to Multi-Agent Systems in Stochastic Environments

Reinforcement learning is one of the more attractive machine learning technologies, due to its unsupervised learning structure and ability to continually even as the operating environment changes. Applying this learning to multiple cooperative software agents (a multi-agent system) not only allows each individual agent to learn from its own experience, but also opens up the opportunity for the individual agents to learn from the other agents in the system, thus accelerating the rate of learning. This research presents the novel use of fuzzy state aggregation, as the means of function approximation, combined with the policy hill climbing methods of Win or Lose Fast (WoLF) and policy-dynamics based WoLF (PD-WoLF). The combination of fast policy hill climbing (PHC) and fuzzy state aggregation (FSA) function approximation is tested in two stochastic environments; Tileworld and the robot soccer domain, RoboCup. The Tileworld results demonstrate that a single agent using the combination of FSA and PHC learns quicker and performs better than combined fuzzy state aggregation and Q-learning lone. Results from the RoboCup domain again illustrate that the policy hill climbing algorithms perform better than Q-learning alone in a multi-agent environment. The learning is further enhanced by allowing the agents to share their experience through a weighted strategy sharing

Fuzzy State Aggregation and Off-Policy Reinforcement Learning for Stochastic Environments

Reinforcement learning is one of the more attractive machine learning technologies, due to its unsupervised learning structure and ability to continually learn even as the environment it is operating in changes. This ability to learn in an unsupervised manner in a changing environment is applicable in complex domains through the use of function approximation of the domain’s policy. The function approximation presented here is that of fuzzy state aggregation. This article presents the use of fuzzy state aggregation with the current policy hill climbing methods of Win or Lose Fast (WoLF) and policy-dynamics based WoLF (PD-WoLF), exceeding the learning rate and performance of the combined fuzzy state aggregation and Q-learning reinforcement learning. Results of testing using the TileWorld domain demonstrate the policy hill climbing performs better than the existing Q-learning implementations

Fuzzy State Aggregation and Policy Hill Climbing for Stochastic Environments

Reinforcement learning is one of the more attractive machine learning technologies, due to its unsupervised learning structure and ability to continually learn even as the operating environment changes. Additionally, by applying reinforcement learning to multiple cooperative software agents (a multi-agent system) not only allows each individual agent to learn from its own experience, but also opens up the opportunity for the individual agents to learn from the other agents in the system, thus accelerating the rate of learning. This research presents the novel use of fuzzy state aggregation, as the means of function approximation, combined with the fastest policy hill climbing methods of Win or Lose Fast (WoLF) and policy-dynamics based WoLF (PD-WoLF). The combination of fast policy hill climbing and fuzzy state aggregation function approximation is tested in two stochastic environments: Tileworld and the simulated robot soccer domain, RoboCup. The Tileworld results demonstrate that a single agent using the combination of FSA and PHC learns quicker and performs better than combined fuzzy state aggregation and Q-learning reinforcement learning alone. Results from the multi-agent RoboCup domain again illustrate that the policy hill climbing algorithms perform better than Q-learning alone in a multi-agent environment. The learning is further enhanced by allowing the agents to share their experience through a weighted strategy sharing

A Method for Revealing and Addressing Security Vulnerabilities in Cyber-physical Systems by Modeling Malicious Agent Interactions with Formal Verification

Several cyber-attacks on the cyber-physical systems (CPS) that monitor and control critical infrastructure were publically announced over the last few years. Almost without exception, the proposed security solutions focus on preventing unauthorized access to the industrial control systems (ICS) at various levels – the defense in depth approach. While useful, it does not address the problem of making the systems more capable of responding to the malicious actions of an attacker once they have gained access to the system. The first step in making an ICS more resilient to an attacker is identifying the cyber security vulnerabilities the attacker can use during system design. This paper presents a method that reveals cyber security vulnerabilities in ICS through the formal modeling of the system and malicious agents. The inclusion of the malicious agent in the analysis of an existing systems identifies security vulnerabilities that are missed in traditional functional model checking

Old-growth forests, carbon and climate change: Functions and management for tall open-forests in two hotspots of temperate Australia

The prognosis and utility under climate change are presented for two old-growth, temperate forests in Australia, from ecological and carbon accounting perspectives. The tall open-forests (TOFs) of south-western Australia (SWA) are within Australia’s global biodiversity hotspot. The forest management and timber usage from the carbon-dense old-growth TOFs of Tasmania (TAS) have a high carbon efflux, rendering it a carbon hotspot. Under climate change the warmer, dryer climate in both areas will decrease carbon stocks directly; and indirectly through changes towards dryer forest types and through positive feedback. Near 2100, climate change will decrease soil organic carbon (SOC) significantly, e.g. by ~30% for SWA and at least 2% for TAS. The emissions from the next 20 years of logging old-growth TOF in TAS, and conversion to harvesting cycles, will conservatively reach 66(±33) Mt-CO2-equivalents in the long-term – bolstering greenhouse gas emissions. Similar emissions will arise from rainforest SOC in TAS due to climate change. Careful management of old-growth TOFs in these two hotspots, to help reduce carbon emissions and change in biodiversity, entails adopting approaches to forest, wood product and fire management which conserve old-growth characteristics in forest stands. Plantation forestry on long-cleared land and well-targeted prescribed burning supplement effective carbon management

Carbon management of commercial rangelands in Australia: Major pools and fluxes

Land-use emissions accompanying biomass loss, change in soil organic carbon (Delta SOC) and decomposing wood-products, were comparable with fossil fuel emissions in the late 20th century. We examine the rates, magnitudes and uncertainties for major carbon (C) fluxes for rangelands due to commercial grazing and climate change in Australia. Total net C emission from biomass over 369 Mha of rangeland to-date was 0.73 (+/- 0.40) Pg, with 83% of that from the potentially forested 53% of the rangelands. A higher emission estimate is likely from a higher resolution analysis. The total Delta SOC to-date was -0.16 (+/- 0.05) Pg. Carbon emissions from all rangeland pools considered are currently 32 (+/- 10) Tg yr(-1) -equivalent to 21 (+/- 6)% of Australia's Kyoto-Protocol annual greenhouse gas emissions. The Delta SOC from erosion and deforestation was -4.0 (+/- 1.6) Tg yr(-1) -less than annual emissions from livestock methane, or biomass attrition, however it will continue for several centuries. Apart from deforestation a foci of land degradation was riparian zones. Cessation of deforestation and onset of rehabilitation of degraded rangeland would allow SOC recovery. If extensive rehabilitation started in 2011 and erosion ceased in 2050 then a Delta SOC of -1.2 (+/- 0.5) Pg would be avoided. The fastest sequestration option was maturation of regrowth forest in Queensland with a C flux of 0.36 (+/- 0.18) Mg ha(-1) yr(-1) in biomass across 22.7 Mha for the next 50 yr; equivalent to similar to 50% of national inventory agriculture emissions (as of mid 2011): and long-term sequestration would be 0.79 (+/- 0.40) Pg. Due to change in water balance, temperature and accompanying fire and drought regimes from climate change, the forecast Delta SOC from the forested rangelands to 0.3 m depth was -1.8 (0.6) Pg (i.e. 38 (12)% of extant SOC stock) resulting from a change in biomass from 2000 to 2100. For improved management of rangeland carbon fluxes: (a) more information is needed on the location of land degradation, and the dynamics and spatial variation of the major carbon pools and fluxes; and (b) freer data transfer is needed between government departments, and to the scientific community

Are there any circumstances in which logging primary wet-eucalypt forest will not add to the global carbon burden?

Uncertainty associated with past land-use emissions restricts quantification of climate change effects. We identify the major affects of commercial forestry initiated over recent decades on Tasmanian primary-forest carbon (C), and search for means to mitigate its ongoing impacts. Spatio-temporal trends were derived from records of commercial operations combined with biomass data. Over the last two decades, the majority of forest C destined for short- or long-term emission (LTE, i.e. over several centuries and multiple harvests) was from clearfelling the higher-biomass wet-eucalypt forests on public land. Carbon dynamics at the unit-area-level for logging two disparate, wet-eucalypt forests were modelled. Parameters were varied to determine management options and model sensitivities under conversion by clearfell and intense burn to either eucalypt plantation or forest regeneration from local eucalypt seed. The first cycle of conversion of primary-forests contributed 43(±5)% to the LTE, and the LTE constituted ∼50% of the primary-forest C stock. Whether the first logging of even-aged primary-forests was prior to or after maturity, the LTEs were equivalent, although short-term emissions (STEs) were ∼2× higher from old-growth. Minor variations in soil organic carbon efflux during operations significantly altered LTEs. Conversion of wet-eucalypt by clearfell from 1999 to 2009 incurred an LTE of 2(±1.6) Tg from each year's logging. Lengthening the harvesting interval for sown forests from 80 to 200 years reduced LTEs by 42% and eucalypt wood-products by 26%; but yielded 40(±20) Mg ha−1 of C in rainforest understorey—helping to sustain mixed-forest ecosystems and their products. Using 200-yr cycles for the wet-eucalypt already clearfelled could avoid LTEs of ∼15 Tg. Long-term C dynamics under harvest cycles were constrained by mathematical precepts that facilitate climate change modelling, e.g. the time to reach the harvesting-cycle's asymptote is correlated to the half-life of the longer-lived C pool. Emissions are not recovered by sequestration in wood-products unless their half-lives are ∼10× contemporary values—requiring 200–1000 years for recovery, during which time emissions would augment global climate change. Emissions can be reduced by product substitution, and by recycling wood-products, in a stable wood market. Primary-forest is part of a global commons. Comprehensive C accounting cannot occur if logging effects are omitted

Accounting for space and time in soil carbon dynamics in timbered rangelands

Employing rangelands for climate change mitigation is hindered by conflicting reports on the direction and magnitude of change in soil organic carbon (ΔSOC) following changes in woody cover. Publications on woody thickening and deforestation, which had led to uncertainty in ΔSOC, were re-evaluated, and the dimensional-dependence of their data was determined. To model the fundamentals of SOC flux, linked SOC pools were simulated with first-order kinetics. Influences from forest development timelines and location of mature trees, with a potential for deep-set roots, were considered. We show that controversy or uncertainty has arisen when ΔSOC data were not measured along sufficient lengths of the three Cartesian axes and the time axis, i.e. in 4D. Thickening and deforestation experiments have particularly neglected factors affecting the time and depth axes, and sometimes neglected all four axes. Measurements of thickening must use time-spans beyond the calculable breakeven date - when thickening just recovers the SOC lost through land degradation: then all ecosystems are likely to incur net sequestration. The similarity between half-life of carbon pools, and the half-time required for sequestration, mandates that millennial time-spans must be considered in design of SOC experiments. Spatial and temporal averaging of ΔSOC data that accounted for environmentally dependent decomposition rates, revealed that deforestation to pasture incurred a higher and longer-term net emission than earlier reported. Published reports on thickening or deforestation appear no longer contradictory when one considers that they only presented views from lengths of the 4D axes that were too limited. Adoption of this understanding into carbon accounting will allow more precise estimates of carbon fluxes for emission trading schemes and national reports